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ABSTRACT 

In this paper we used, the Dai-Yuan nonlinear conjugate gradient (DYCG) method, to the three-term 
conjugate gradient (TDYCG) method and the derivation of the method, based on the Perry’s conjugacy condition .The 
global convergence was proved, with Wolfe line search . Numerical experiments are reported and shows the presented 
method outperforming, some other three-term CG methods. 
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INTRODUCTION 

We consider the unconstrained optimization problem: 

min/ (a), x<eR" (l.i) 

Where f '. R —> R is a continuously differentiable function and bounded below the nonlinear 
conjugate gradient method is an iterative method whereby at the [k +1) iteration X k +1 is given by: 
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X k+1 = X k +a k d k 


( 1 . 2 ) 


Where d /, denotes the search direction and a k is the step length. The search direction d ^ + i is calculated 
by 

d k +1 — — 8k +i Pk +\ d k ^ , k=0,l,.... (1.3) 

In (1.3) the quantity g k = V/ {x k ) denotes the gradient of/* atV k , while //. +J is scalar which 
can be defined by 
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Or by other formula (e.g. see [l,2,7]).The corresponding method is respectively called FR(Fletcher- 
Revees[6]),DY(Dai-Yuan[3]),CD(conjugate-Decent[7]),PR(Polak-Ribiere[l l]),HS(HestenesStiefel[9]),respectively. 


Where 


means the Euclidean norm. The line search in conjugate gradient algorithms is often based on the general Wolfe 


conditions [13,14] 

f (x k + a k cl k )</(**) + Sa k d k 


gl + 4 k *<7gk d k 


(1.5) 


( 1 . 6 ) 


Where d k is a descent direction and 0 <5 ^ (T'v 1 .However, for some conjugate gradient algorithms, 
stronger Wolfe (SW) condition used, defined by equation (1.5) and: 


sl + 4k ^-VgJdk 


Are needed to ensure the convergence and to enhance the stability. 
The pure conjugacy condition is represented by the form 

rT 


(1.7) 


d k+l y k =o 


( 1 . 8 ) 


For nonlinear conjugate gradient methods .The extension of the conjugacy condition was studied by Perry [10], he 
tried to accelerate the conjugate gradient method by incorporating the second-order information into it specifically, he used 
the secant condition 


H k + Jk = S l 


(1.9) 


Of quasi-Newton methods, where asymmetric matrix H k +1 is an approximation to the inverse Hessian. For 
quasi-Newton methods, the search direction d k+l can be calculated in the form 


dk+ i H k+\S k+\ 

By (1.9) and (1.10) the relation 


d Ly k = ~( H k +l s k+ 1 ) T y k = -§L ( H k+l y k ) = sL s k 


( 1 . 10 ) 


(lid 


Holds By taking this relation into account. Perry replaced the conjugacy condition (1.8) by the condition 

d k+{y k ~~§k+l S k (1.12) 

Recently Zhang, Zou and Li [15, 16], proposed the following three-term conjugate gradient methods 
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d k+l =-g k+x +p m d k -fg k 
d k+x =-g k+l +(3 PR d k ~f ] g k 
d k ^=-g k+l +P m d k -f ] g k 
d k+ 1 =~g k+ 1 +P m d k +^ 4) g k 


(1.16) 


(1.13) 


(1.14) 


(1.15) 


Where 



(1.17) 


The rest of the paper is organized as follows: In section 2, we presented a new three-term conjugate gradient 
(TKLCG) method, which was obtained by modification of the Khalil-Linda CG method. Section 3, presents the global 
convergence of the proposed algorithm. In section 4, some numerical results and comparisons with some other three-term 
conjugate gradient methods, were presented. 

NEW THREE-TERM CG-METHOD (TKLCG) 

In this section, we derive a new three-term conjugate gradient algorithm, for unconstrained optimization which is 
generalization to, the Dai-Yuan CG method. Consider the following three-term search direction 



( 2 . 1 ) 


If we use inexact line search and Perry conjugacy condition (1.12), we get from (2.1) the following value for 




(2.2) 


Therefore our new three-term (TKLCG) conjugate gradient algorithm can be defined as 





T T 


(2.3) 


v Sk+iy/cj 


We see from (2.3), for quadratic convex function with exact line search the above search direction reduces to the 
Fletcher CD method i.e. 



(2.4) 
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(TKLCG) Algorithm 

Stepl. Given A , E. R ,£ ~> 0, d l = — g k 

Step2. If ||g k + j || — £ , stop, else go to Step 3 


Step3. Find 0C k satisfying Wolfe condition (1.5) and (1.6). 

Step4. Compute a new iterative % k +\ by A ~A k + CC k d k . 


set 


Step5. Compute r . n +1 and 9 k +| from (2.2 )d k+{ — 6 k+l g k+l +f3 S k + y^ +] y k and 
k=k+l go to step 2. 

3. DESCENT PROPERTY AND GLOBAL CONVERGENCE ANALYSIS 

Next we will show that our three-term CG (2.3)-method satisfies the descent property and global converges. 

Theorem 3.1.Let {a k } and {d k } be generated by the equation (1.2),(2.3)and OC k satisfies Wolfe line search 
conditions (1.5) and (1.6), then d k g k <0 hold for all k ^ 1 . 


Proof. The conclusion can be proved by induction. When k — 1 , we have d i 8i f <° 


Suppose that d k g k <0 hold for all k. From (2.3) we have 
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By Wolfe condition y k^k > 0 and let 0 < £ +I k ^ 1, then we can deduced that 

Sk+dk y k *k 

d k +\8 k +l < 0 holds for all k ^ 1 . 

To prove the global convergence we need the following assumption [8] 

HI. The objective function f is bound below in the level set £2 = { X G where 

X j the starting point is. 


H2 .f Is continuously differentiable in a neighborhood N of a and its gradient g is Lipchitz continuous, 
there exist L >0 such that 


g(*)-g(} ; )||^£||* -y I Vx,y g N 


(3.1) 


H3. f is uniformly convex function, then there exists a constant fi> 0 such that 

yl'k*Mht and v\\ s S^yl s k ^ L lkf 


On the other hand, under assumption [10], it is clear that there exist positive constant B such 

\\x II < B ,\/x E £2(3.3) 


l<\\g(x)\\<y ,\/x e £2 (3, 


4) 


(3.2) 


Lemma (1) [8] 

Suppose that Assumption [10] and equation (3.3) hold .consider any conjugate gradient method in from (1.2) and 
(1.3), where d k is a descent direction and ^ is obtained by the strong Wolfe (SW) .If 
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z 


k >1 \\d 


(3.5) 


k+ 1| 
Then we have 


!™( inf lk t |)=°- 


More details can be found in [l,4and 12] 

Theorem (1) 

Suppose that, assumption [8], equation (3.3) and the descent condition hold. Consider a conjugate gradient 
method in the equation (2.3) where CX K is computed from Wolfe line search condition (1.5) and (1.6) If the objective 

function is uniformly on set Q, then HlTl ^ iflf ||g k || j — 0 . 


Proof 


d k + i=-0k + i8 k+ i + fir s k +7ktiy, 
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Using Cauchy Schwartz together with inequality (3.3) we get 
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By Using Assumption [8] then we get 
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^F=e tt ,(r) 2 +Mf(ff' +jC' i) 


then we get 
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By Using Lemma (1) then we get 

lim (inf 18,1) = ° 


(3.15) 


4. NUMERICAL RESULTS AND COMPARISONS 

In this section, we compared the performance of new formal 0 k +1 , and developed a new three-term method of 

conjugate gradient method, to other classical conjugate gradient method Dai-Yuan (DY) and Khalil-linda (KL1) 
algorithms). We have selected 20 large scale unconstrained optimization problems, for each test problems taken from 
Andrie, (2008) [2], For each test function, we have considered numerical experiments with the number of variables 
n=100,..., 1000 . These two new versions were compared with well-known conjugate gradient algorithm, Dai-Yuan (DY) 
and Khalil-linda (KL1) algorithms. All these algorithms were implemented with standard Wolfe conditions (1.5) and (1.6). 

In all these cases, the stopping criteria is thelg^ || = 10 . All codes were written in double precision, FORTRAN 

language, with F77 default compiler settings. The test functions usually start the point standards initially summarizing 
numerical results for 730 functions, recorded in the Table (4.1). The results for the 730 functions, explained in Figure (1), 
(2) and (3), the performance profile, by Dolan and More' [5] was used, to display the performance of the Generalized Dai- 
Yuan Nonlinear Conjugate Gradient algorithm, with Dai-Yuan (DY) and Khalil-linda (KL1) algorithms. Define P=730 as 


the whole set of ^p test problems and S=3 the set of the interested solvers. Let l ps 
function evaluations, required by solver S for problem P . Define the performance ratio as 

/. 


be the number of objective 


p,s 


p,s 


i 


(4.1) 


Where l p — S G S } . It is obvious that ^ p s — 1 for fill p,S-if a solver fails to solve a 

Problem, the ratio v? is assigned to be a large number M. The performance profile for each solver S is defined as the 

y 

following cumulative distribution function for performance ratio 1 p ,s , 
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(4.2) 


Obviously, P s l 1 ; represents the percentage of problems, for which solver S is the best. See figure [5] for more 

details, about the performance profile. The performance profile can also be used to analyze, the number of iterations, the 
number of gradient evaluations and the cpu time. Besides, to get a clear observation, we give the horizontal coordinate, a 
log-scale . 

The numerical results show the efficiency comparisons, with some other three-term conjugate gradient methods. 



Figure 1: Comparison Based on Iteration 
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Figure 2: Comparison Based on Function 
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Figure 3: Comparison Based Ontime 
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